SUBREPRESENTATION THEOREM FOR 
p-ADIC SYMMETRIC SPACES 



SHIN-ICHI KATO AND KEIJI TAKANO 

Abstract. The notion of relative cuspidality for distinguished 
representations attached to p-adic symmetric spaces is introduced. 
A characterization of relative cuspidality in terms of Jacquet mod- 
ules is given and a generalization of Jacquet 's subrepresentation 
theorem to the relative case (symmetric space case) is established. 



Introduction 

Let G be a reductive p-adic group, o an involution on G and H 
the subgroup of all cx-fixed points in G. An admissible representa- 
tion (tt,V) of G is said to be H- distinguished if the space (V*) H of 
all if-invariant linear forms on V is non-zero. Such a representation 
arises as a local component of automorphic representations which are 
of particular interest (see [J] for example). From representation theo- 
retic point of view, distinguished representations are the basic object of 
harmonic analysis on the p-adic symmetric space G/H. By the Frobe- 
nius reciprocity, these representations can be realized in the space of 
smooth functions on G/H. The classification or parametrization of 
such representations would be a fundamental problem. Over the real 
field, harmonic analysis on semisimple or reductive symmetric spaces 
has been fully developed since 1980's (see [0] and [D]). By contrast, 
the theory over p-adic fields has not been developed yet. 

In this paper, we suggest a new basic tool for the study of harmonic 
analysis on p-adic symmetric spaces, and establish the relative version 
(for general p-adic symmetric spaces) of Jacquet 's subrepresentation 
theorem (for general p-adic groups) . 

Jacquet's subrepresentation theorem asserts that any irreducible ad- 
missible representation of a reductive p-adic group can be embedded 
in a parabolically induced representation. The inducing representation 
of a Levi subgroup can be taken as an irreducible cuspidal one, that 
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is, a representation whose matrix coefficients are compactly supported 
modulo the center. To establish the relative version of this theory, 
first we introduce the notion of relative cuspidality as follows. An H- 
distinguished representation of G is said to be H -relatively cuspidal 
if all the generalized matrix coefficients defined by if -invariant linear 
forms (which we shall call H-matrix coefficients) are compactly sup- 
ported modulo the product of H and the center of G. We use only 
such ones as the inducing representations. Besides, as the inducing 
subgroups, we use only a particular class of parabolic subgroups. A 
parabolic subgroup P is said to be a -split if P and cr{P) are opposite. 
In this case, M = P H o~(P) is a cr-stable Levi subgroup of P and the 
quotient Mj (M D H) gives a symmetric space of lower rank. Now we 
state the main theorem of this paper. 

Theorem A (Theorem 17. ip . Let n be an irreducible H -distinguished 
representation of G. Then there exists a a -split parabolic subgroup P of 
G and an irreducible M T\H -relatively cuspidal representation p of M = 
P fl cr(P) such that tt can be embedded in the induced representation 
Ind£(p). 

This gives a generalization of Jacquet's subrepresentation theorem 
in the following sense. Take an arbitrary reductive p-adic group Gi and 
form the direct product G = G± x G\. Let o be the involution on G 
which permutes factors. Then the cr-fixed point subgroup in G is the di- 
agonally embedded subgroup AGi. An irreducible AGi-distinguished 
representation of G\ x G\ is of the form 7Ti®7fi where 7i"i is an irreducible 
admissible representation of G\ and 7Tl is the contragredient of ~K\. It is 
AGi-relatively cuspidal if and only if 7Ti is cuspidal in the usual sense. 
A er-split parabolic subgroup of G\ x G\ is of the form Pi x Pf where 
Pi and Pi are opposite parabolics of G\. Now, Theorem A applied 
to this situation will give an embedding of any irreducible admissible 
representation 7i"i £§) 7Tl into 

Ind (pi ®pi)~ Indg ( Pl ) ® Ind^ (p\) 

where p\ is an irreducible cuspidal representation of a Levi subgroup of 
Pi. In this way, Jacquet's subrepresentation theorem can be recovered 
from our theorem. We have an embedding tci <^-> Indp^pi) on the first 
factor, and also tt\ Ind^i (pi) on the second factor at the same time. 

In the process of obtaining Theorem A, we refer to Casselman's proof 
of Jacquet's subrepresentation theorem as a prototype. In [C], a canon- 
ical pairing of Jacquet modules is constructed. It provides a relation 
of the asymptotic behaviors between matrix coefficients for a given 
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representation and those for its Jacquet modules. By this relation, 
the well-known characterization of cuspidality is shown: An admissible 
representation is cuspidal if and only if the Jacquet modules along all 
proper parabolics vanish. After this characterization, Jacquet's subrep- 
resentation theorem readily follows from the inductive argument using 
the Frobenius reciprocity. 

We consider a relative version of Casselman's canonical pairing, which 
is expected to be a basic tool for harmonic analysis of G/H. Let P be 
a ex-split parabolic subgroup with the Levi subgroup M = P fl cr(P). 
Let (irp, Vp) denote the Jacquet module of (tt, V) along P. We shall 
construct a mapping 

r P : (V*f -> (V£) MnH 

of invariant linear forms. This will provide a relation of the asymp- 
totic behaviors between if -matrix coefficients for 7r and M fl if -matrix 
coefficients for irp. Using this relation, we can deduce another main 
theorem which gives a characterization of relative cuspidality: 

Theorem B (Theorem 16.91) . An H- distinguished representation (tt, V) 
ofG is H -relatively cuspidal if and only ifrp((V*) H ) = for any proper 
o~ -split parabolic subgroup P of G. 

Theorem A is a natural consequence of this characterization. 

This paper is organized as follows. Section [1] is devoted to definitions, 
notation and some elementary properties of relatively cuspidal repre- 
sentations. In section [2J basic notation and properties concerning with 
tori, roots and parabolic subgroups associated to the involution are 
prepared. Section [3] deals with the analogue of Cartan decomposition 
for p-adic symmetric spaces given independently by Benoist-Oh [BOJ 
and Delorme-Secherre |DS] . This will be used in an important step of 
the analysis of ii-matrix coefficients. In section HI certain families of 
open compact subgroups are introduced. Lemma 14.61 on a property of 
the subgroups in such a family will be a key to the construction of the 
mapping rp in section After that, we give a result Proposition 15.51 
on an asymptotic behavior of ii-matrix coefficients. This will be used 
repeatedly in the proof of various properties of rp such as the transi- 
tivity (Proposition 15.91) . Section [6] is devoted to the proof of Theorem 
B (Theorem 16.91) . The proof of Theorem A (Theorem 17.11) is given 
shortly in section [7J Finally, we give several examples of relatively 
cuspidal representations in section [HJ 
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After this paper was completed, we learned that Nathalie Lagier 
has obtained independently the same result as our Proposition 15.51 and 
Proposition 15.61 (1). which has appeared in the announcement [L]. We 
are grateful to Patrick Delorme for pointing this out. 

1. Relatively cuspidal representations 

In this section, we shall introduce the notion of relative cuspidality 
for distinguished representations and give some elementary properties 
of them. 

1.1. Let F be a non-archimedean local field with the normalized abso- 
lute value and the valuation ring Of- We assume that the residual 
characteristic of F is not equal to 2 throughout this paper. Let G be 
a connected reductive group over F and a an F- involution on G. The 
F-subgroup {h G G | a{h) = h} of all cx-fixed points of G is denoted by 
H. Let Z be the F-split component of G, that is, the largest F-split 
torus lying in the center of G. Note that Z is a-stable. The group 
G(F) of F-points of G is denoted by G. Similarly, for any F-subgroup 
R of G, we shall write R = R(F). 

1.2. Let (71", V) be an admissible representation of G over C. It is 
said to be H -distinguished if the space (V*) H of all F-invariant linear 
forms on V is non-zero. For A G (V*) and v G V, let tp\ tV denote the 
function on G defined by 

f\,v(x) = (\,ii(x' 1 )v) (xeG). 

We call such functions (F, A) -matrix coefficients of tt. These are not 
the usual matrix coefficients of ir, since F-invariant linear forms are 
not smooth in general. Let C°°(G/H) denote the space of all right 
F-invariant locally constant C-valued functions on G, on which G 
acts by the left translation. Any subrepresentation of C°°(G/H) is 
F-distinguished by the linear form (p i— > ip(e). All the (F, A)-matrix 
coefficients belong to C°°(G/H) with the obvious G-equivariance 

for x, g e G. Set T x (v) = (p x>v for A G {V*) H and v G V. Then the 
mapping T\ : V — > C°°(G/H) is a G-morphism, which is non-zero if 
and only if A ^ 0. 

1.3. Definition. An F-distinguished representation (ir,V) is said to 
be (F, A) -relatively cuspidal (for A G {V*) ) if all the (F, A)-matrix 
coefficients of n are compactly supported modulo ZH. A representation 
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(tt, V) is said to be H -relatively cuspidalii it is (H, A)-relatively cuspidal 
for all A G (V*) H . 

1.4. Let C%°(G/H) denote the subspace of C°°(G/H) consisting of all 
the functions ip G C°°(G/H) which are compactly supported modulo 
ZH . It is stable under G from the left. An if-distinguished represen- 
tation (jr, V) is (if, A)-relatively cuspidal for A G (V*) H if and only if 
the image T\{V) is contained in Cq°(G/H). 

1.5. The group case. Here we see that the notion of relative cuspidality 
introduced above includes the usual cuspidality as a special case. Take 
a connected reductive F-group Gi and let G be the direct product 

G = d x d 

equipped with the involution 

0(91,92) = (92, 9i)- 
The cr-fixed point subgroup H is the diagonally embedded subgroup 
AG 1 = {(g,g)eG 1 xG 1 \geG 1 }. 

The mapping (^1,^2) l— * 9\9 2 X induces a G-equivariant isomorphism 
of the quotient G/H = (Gi x Gi) /AGi onto the underlying variety 
of Gi on which G = Gi x Gi acts by multiplication from both sides. 
The F-split component Z of G is of the form Z 1 x Zi where Zi is the 
F-split component of Gi. Henceforth, this situation is referred to as 
the group case. 

Any irreducible admissible representation of G — G± x G\ is of the 
form 7Ti ® ir[ where 7i\ and n[ are irreducible admissible representations 
of G\. It is A (^-distinguished if and only if 7r[ is isomorphic to the 
contragredient ifi of m. The natural pairing (, )~ X7ri of ifi and ~K\ 
defines a non-zero AGi-invariant linear form, say A G ((tti <S> tti)*) \ 
which is unique up to constant. It is given by 

(A,ui <8)55i) = (ui,ui)srx W i 

for fi G Vi and Vi G Vi. Observe the relation 

(A, (tti ®TT 1 )(gi,g 2 y 1 (v 1 <8>55i)) = (A, 7Ti(^f ® tTi^ 1 )^) 

for gi, g 2 G Gi and v\ G Vi, G V\. As is seen from this, the (ACi, A)- 
matrix coefficients are identified with the usual matrix coefficients of 
7Ti through the mapping (gi,g 2 ) ^ gig^ 1 - 

Recall that an admissible representation is said to be cuspidal if 
all the usual matrix coefficients are compactly supported modulo the 
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center. Now it is obvious from the above identification that tt\ (g> fti is 
(AG i, A)-relatively (actually AGi-relatively) cuspidal if and only if tt\ 
is cuspidal representation of G\. 

1.6. For an admissible representation (it, V) of G and a quasi-character 
to of Z, put 

V w = {v G V | 7r(2;)u = u;(2;)t> for all 2; G Z}. 

We call a subrepresentation of V an ^-subrepresentation if it is con- 
tained in V u , and call V an ^-representation if V — V u . 

1.7. Lemma. Lei (7r, V) fre an admissible representation of G of finite 
length. Then there exists a non-trivial quotient representation of V 
which is isomorphic to an to -subrepresentation of V for some quasi- 
character uj of Z . 

Proof. Recall from [0, 2.1.9] the direct sum decomposition V = @ w V^ jCX 
where u runs over a finite set of quasi-characters of Z, 

V u>n = {v G V I (tt(z) - uj(z)) n v = for all z e Z) 

and 




It is enough to consider the case V = K, )00 for some quasi-character u 
of Z. If V — Vu,i = V u , there is nothing to prove. If not, there exist 
elements zq G Z and v G V such that 

tt(z )v - uj(z )v ^ 0. 

Set 0i = 7t(z ) — u(z )idv '■ V — > V. This is a non-zero G-morphism 
with non-trivial kernel. Let V\ be the image of 0i. It is a proper G- 
submodule of V. If V\ C V^, then V/Ker(^ 1 ) ~ V\ will give the desired 
quotient. If V\ (£. V u , there are z\ G Z and t> 1 G V\ such that 

7t(^i)t;i - u)(z\)v\ ^ 0. 

Set 02 = (tt(zi) — u(zi)idy) \ Vl : Vj — > Vi. Again this is a non-zero 
G-morphism with non-trivial kernel. Let V2 be the image of 02 > which 
is a proper G-submodule of V\. In this way, we can construct a non- 
zero proper G-submodule Vk of Vk-i with a surjective G-morphism 

Vk-i -» Vfc inductively as long as Vfc_i is not contained in K,. We 
obtain a decreasing sequence 

of G-submodules, together with surjective G-morphisms 

T/ 01 T/ 02 T/ ^ T/ 
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We must have ^ C K, for some k by the finiteness of length of V. 
Now V/Ker((p k o • • • o X ) ~ will give the desired quotient. □ 

1.8. For a quasi-character u of Z which is trivial on Z (1 H, let 
C^(G/H) denote the space of all <p G C£°(G/H) satisfying 

ip(zg) = u^^ifig) 

for g G G and 2 G ^. If an ^-representation is if-distinguished, then 
we must have u\zr\H = 1- An if -distinguished (^-representation (ir, V) 
is (H, A)-relatively cuspidal for A G (V*) H if and only if the image 
T\(V) is contained in G ~ (G/#). 

If uj is unitary, then any subrepresentation of C™ U {G/H) is pre- 
unitary by the G-invariant hermitian inner product 

«^V}>=/ v(g)W)dg (v,^eC™(G/H)) 

Jg/zh 

where dg denotes a G-invariant measure on the quotient G/ZH . 

1.9. Let X(G, M* ) be the set of all positive real valued quasi-characters 
of G and put 

X(G/H, R* ) = { x G X(G, M* ) \ X \h = 1 }■ 

We define X(Z, M> ) and X{Z/Z n M> ) similarly. 

1.10. Lemma. (1) The restriction map 

X(G, R* ) X{Z, R* ), res(x) = x\z (x e *(G, R* )) 
is bijective. 

(2) T/ie above restriction map sends X(G/H, M> ) bijectively onto 
X(Z/ZDH,R* ). 

Proof. (1) is well-known (e.g., (CJ 5.2.5]). For (2), first it is obvious 
that xU G <*W^ n H,R* Q ) for all x e X(G/H,R$ ). On the other 
hand, for a given u; G X(Z/Z D if, M> ), let us take the element x G 
#(G,M£ ) such that xU = ^ by (1). Since z<r(z) G Z n If for all 
z G Z, we have co>(,2cr(z)) = 1 so that o a = uj^ 1 . Thus we have 
(x°o")U = (xlz) ocr = (xU) -1 = X _1 U, which shows that x°cj = X -1 
on G by (1). This proves that x\h = 1 since x(h) = £ ^>o f° r 

all ft. G if . □ 

1.11. Proposition. v4ny finitely generated (H, X) -relatively cuspidal 
representation of G has a non-trivial H- distinguished irreducible quo- 
tient. 
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Proof. Let (tt, V) be a finitely generated admissible representation with 
a non-zero A G (V*) H such that T X (V) C Cfi°{G/H). The quotient 
y/Ker(T A ) is finitely generated, hence is of finite length by [Cj 6.3.10]. 
Apply Lemma [1771 to obtain a quotient (Tf, V) of V/Ker(T\), hence of V, 
which is isomorphic to an cu-subrepresentation of V/Ker(T\) for some uj. 
Then (if, V) is regarded as a subrepresentation of C^^G/H) through 



V/Ker(T\) ~ T A (F) C C?(G/H). Let 3%) be the quasi-character of 
Z defined by 



for z G Z. This belongs to X(Z/Z n #,JR> ) and $M := w3*M _1 
is unitary. Now, take the element x £ X(G/H, M>o) such that xU = 
5i(u;) _1 by Lemma [1.101 (2) and consider the representation x ' tt on V 
defined by 

(x ■ 7f) (sO = x{g)K{g) 

for g <E G. It is regarded as a subrepresentation of C^^(G/H), which 

is pre-unitary by 11.81 Thus (x ■ 7f, V) is decomposed into a direct sum 
of finitely many irreducible subrepresentations (see [UJ 2.1.14]). The 
decomposition for the action x ' also yields that for the action 7T, 
hence the claim follows. □ 



In this section, we shall prepare basic notation and properties con- 
cerning with tori, roots and parabolic subgroups associated to p-adic 
symmetric spaces. 

2.1. An F-split torus S in G is said to be (cr, F)- split if a(s) = s -1 for 
all s G S. Take a maximal (cr, F)-split torus So of G and a maximal 
F-split torus A0 of G containing S . Then A0 is a-stable by |HW[ 4.5]. 
Put M = -Zg(So) an d M0 = Zg(Aq). Clearly these are also cr-stable. 

Let Xp(A^) be the free Z-module of all F-rational characters of A0 
on which a acts naturally. Let $ = $(G, Aq) be the root system of 
(G, A0). Then a leaves $ stable. Set 



This is a closed subsystem of $. As in [HH, 1.6], we choose a o- 
basis A (and the corresponding set $ + of positive roots) of $ so that 
a (a) £ $ + for all a G $ + \ Put A ff = An This gives a 
basis of the subsystem Let be the longest element of the Weyl 
group of $ CT with respect to the basis A a . As in [HH, 1.7], let cr* be 
the involution on Xp(A$) given by cr* = — cr o w^ a . It is known that 
cr* leaves A stable. 




2. cr-SPLIT PARABOLIC SUBGROUPS 



= { a G $ I a (a) = a }. 
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2.2. Let So C A0 and A be as above. Let P0 (d A0) be the min- 
imal parabolic F-subgroup of G corresponding to A. In this paper, 
parabolic F-subgroups containing such P$ are said to be standard with 
respect to (S o ,A0,A). They correspond to subsets of A. Let us fix 
the notation. For a subset / C A, let P/ be the standard parabolic 
F-subgroup of G corresponding to / with the unipotent radical Uf. 
Let Aj denote the identity component of the intersection of all the ker- 
nels of a : A0 — > G m , a G I, and set Mj = Z G (Aj). We have a Levi 
decomposition Pj = M/ x \Jj. The torus Aj is the F-split component 
of Mj. Let P7 be the unique parabolic F-subgroup of G such that 
Pj H P7 = M/ and U7 the unipotent radical of Pj . Later we shall 
drop the subscript I if there is no fear of confusion. 

2.3. A parabolic F-subgroup P of G is said to be a -split if P and er(P) 
are opposite. In such a case, P R cr(P) gives a cr-stable Levi subgroup 
of P. Let us fix So, A0 and A as above. If Pj is cr-split for a subset 
I C A, then we must have a (Pi) = Pj, that is, Pi H a (Pi) = M/ 
(see |HW| 4.6]). Hence we also have er(Uj) = Uj. Further, recall from 
[HHt 2.6] that Pi is cr-split if and only if A CT C / and the subsystem 
<&/ of $ generated by I is cr-stable. There is an alternative description 
for Pj to be cr-split as follows: If / contains A CT , then leaves 
stable so that 

is a-stable is a* (= —a o WA CT )-stable 



We say that a subset of A is a-split if it contains A CT and is cr*-stable. 
Thus Pi is cr-split if and only if J is a cr-split subset of A. 

Note that every cr-split parabolic F-subgroup arises as a standard 
cr-split one for a suitable choice of S , A0 and A. See |HWt 4.6, 4.7] 
and also Lemma [2.51 (1) below. 

2.4. The subset / = A CT is a minimal cr-split subset of A. The cor- 
responding parabolic F-subgroup Pa ct is denoted by Po- This is a 
minimal cr-split parabolic F-subgroup of G and the cr-stable Levi sub- 



group M Act = P ncr(P ) coincides with M = Z G (S ) (see [HWl 4.7]). 
Put P = P Act , U = U A(T and U = U Act . 



2.5. Lemma. Let us fix So, A0 and A as above. 

(1) Any a-split parabolic F-subgroup of G is of the form 7 _1 P/7 for 
some a-split subset I C A and 7 e (M H) (F). 

(2) If (M H) (F) = M H , then any a-split parabolic F-subgroup of 
G is H -conjugate to a standard a-split one. 




(since cr*(A) = A and An$/ = I). 
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Proof. (2) follows directly from (1). For (1), let P C G be any cr-split 
parabolic F-subgroup. As in [HWl 4.9, 4.11], we can take an element 
7 G (P0H)(F) such that 7P7" 1 contains P0 and is cr-split. It is enough 
to see that p~ lr y G (M H)(F) for some p 6 P . Since P0H = PoH by 
[HWl 4.8], we have 

x := 7^(7)^ G (UoMqUo ){F) = U M U . 

Express x as x = UquiqUq with w G Uq, m G M and Uq G Uq . Since 
x = we have 

u m u = a{u Y l a(mQY l (y(uQY l . 

By the uniqueness of the expression of elements of UqMqUq , we must 
have 

m = a(m Q )~ x , u =(j(u o )- 1 . 

Now it is seen that 

u 1 lo-(u = u 1 xa(u ) = u^x^Uq)' 1 = m G M . 

This shows that u^j G (M Q H) (F). □ 

2.6. For each a G A\A CT , we put I a = A\{a, cr*(a)}. (It may happen 
that a = a* (a).) This is a maximal proper cr-split subset of A. Hence 
all the maximal cr-split parabolic F-subgroups of G are of the form 
7~ x P/ a 7 for some a G A \ A CT and 7 G (M H)(F). 

2.7. For a cr-split subset I C A, the identity component of Aj fl S is 
denoted by S7-. We shall call S/ i/ie (a,F)-split component 0/M7 (or 
ofPj). Note that the ( cr, F)-split component Sa ct of the minimal cr- 
split parabolic F-subgroup Pa ct = Po coincides with S , the maximal 
(cr, F)-split torus we choose. 

For a positive real number e ^ 1, set 

Aj(e) = { a G A r \ \a a \ F ^ e (a G A \ J)} 

and 

57(e) = 5 7 n Aj(£) = { s G 5/ I |s Q | F ^ £ (a G A \ /)}. 
Also put 

S+(e) = {s G S^s" 1 g Sy(s)} 

= {s e S I \\s a \ F ^e (ae A\/)}. 
Since 5*7 (e) C ^7(5), the following lemma is apparent (see [UJ 1.4.3]). 

2.8. Lemma. Let I C A be a a-split subset. For any two open compact 
subgroups U\, U2 ofUj, there exists a positive real number e ^ 1 such 
that sU\S~ l C U2 for all s G S^e). 
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Next we shall give a lemma on the modulus character of c-split 
parabolic subgroups. 

2.9. Lemma. Let P = M x U be a a -split parabolic F-subgroup of G. 
Then the modulus character 5p of P is trivial on M fl H. 

Proof. Let A be the F-split component of M. Since Sp is positive real 
valued, it is determined by the values on A according to Lemma [1.101 
(1). It is enough to show that Sp is trivial on A fl H by (2) of Lemma 
11.101 We may suppose that P = Pj for a cx-split subset I C A. Then 
we have 

S P (a)= fl K|™ Q {aeA) 

where m a denotes the dimension of the root space attached to a G $. 
Note that m a = m a f a ) since a maps the root space attached to a 
isomorphically onto the one attached to a (a). If further a G A fl H, 
we have 

5 2 (a) = 5(a)5(a(a))= J] \a a \^ \{ |a«*»>£" 

= n i a T a n K (a) ir <a) 
= n n Kir^ 1 

and the claim follows. □ 

3. The analogue of Cartan decomposition 

We shall recall from [BOj and |DS] a certain decomposition theorem 
related to p-adic symmetric spaces and give a variant of it for our later 
use. It will play an important role in the study of the support of H- 
matrix coefficients. 

3.1. Let us fix a triple (S , A0, A) as in 12.11 For simplicity, put Sq = 
(1) so that 

Sq = { s g So(F) I \s a \ F ^ 1 (a G A \ A,)}. 

Let {Sq }jej be a set of representatives for if-conjugacy classes of 
maximal (a, F)-split tori of G. In this case, the index set J is finite by 
[HH[ 2.12]. For each j G J, there exists an element yj G (MqH) (F) 
such that y^Soy,- = S ( j) by (HWl 10.3]. Set 

Wb(S?>) = N G (S^)/Z G (S^) 
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and 

W H (S^)=N H (S^)/Z H (SH ) ). 

Naturally Wh(Sq ) is regarded as a subgroup of the finite (Weyl) group 
Wg(Sq^)- Let Mj C Nq(Sq ) be a complete set of representatives of 

3.2. Theorem (|DS, Theorem 3.10]). There exists a compact subset Q 
of G such that 

G=U U SlS+y 3 nH. 

An essentially equivalent assertion is given also in [BOl Theorem 1.1]. 

3.3. Now for each j G J, we have 

N a (8$>) = Naiyj'Soyj) = y^N G (S )y r 

Put 

We have Afj C N G (S ) for all j G J. The above decomposition is 
written as 

G=U U ttS+ny 3 H. 

Furthermore, we have ncr(n) _1 G Mo(-F) for any n G N g (Sq) since 
no~{n)~ l centralizes So- Consequently for any n G N G (So) and y G 
(M H) (F), we have 

nya(ny)~ l = r^yer^) -1 )^) -1 

= n{ya{y)^ 1 )rT 1 ■ na^n)^ 1 G M . 

This means that ny G (M H) (F). As a result, we have a variant of 
Theorem 13.21 in the following form. 

3.4. Corollary. There exists a compact subset Q of G and a finite 
subset T of (M H) (F) such that 

G = QS+TH. 

4. Preliminaries on open compact subgroups 



For a later use on the study of invariant linear forms on Jacquet mod- 
ules, we shall construct a particular family of open compact subgroups 
adapted to the involution. 
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4.1. In this section, all parabolic F-subgroups are standard ones with 
respect to some fixed data (S o ,A0, A) as in 12.31 but the subscripts / 
(c A) will be omitted. For a parabolic F-subgroup P = M x U of G 
and an open compact subgroup K of G, we set 

U K = UnK, M K = MDK } U K = U-f]K. 

Note that a(Mx) = Mk and <j(Uk) = if K is cr-stable and P is 
cr-split. 

4.2. For each choice of a maximal F-split torus A0 and a basis A of 
the root system of (G, A0), there is a decreasing sequence {K' n } n > of 
open compact subgroups of G satisfying the following properties as in 
1-4.4]: 

(1) It gives a fundamental system of open neighborhoods of the 
identity em. G. 

(2) For each n ^ 1, the subgroup K' n is normal in K' Q and the 
quotient K' n /K' n+1 is a finite abelian p-group where p denotes 
the residual characteristic of F. 

(3) For each K' = K' n (n ^ 1) and each standard parabolic F- 
subgroup P = M x U of G (corresponding to (A0,A)) with 
the F-split component A, the product map 

U^, x M K , x U K > -> K' 

is bijective and 

a ■ Uk> ■ a 1 C Uk', cl~ 1 • U^, ■ a C U^, 

for all a G A~(l). 

(4) For each standard parabolic subgroup P = M x U of G, the 
family {Mr\K' n } n > enjoys the same properties as ([I])-© above 
for the group M. 

Here the latter half of ([2]) is not apparent in [Q, 1.4.4]. However, in 
the F-split case, the argument in [G] 2.2.11] shows that each quotient 
K' n /K' n+1 is isomorphic to the additive group of a Lie algebra over the 
residue field of F. The general case is reduced to the F-split case as in 
1.4.4]. 

4.3. Lemma. Fix a data (So, A0, A) as in \2.1\ and let {K' n } n > be as 
above. Put K n = K' n na(K' n ) for each n. Then the family {K n } n > is a 
decreasing sequence of a -stable open compact subgroups of G satisfying 
the following properties: 

(1) It gives a fundamental system of open neighborhoods of the iden- 
tity e in G. 
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(2) For each n ^ 1, the subgroup K n is normal in Kq and the 
quotient K n /K n+ i is a finite abelian p- group. 

(3) For each K = K„ (n ^ 1) and each a -split parabolic subgroup 
P = M x U of G (standard with respect to (So, A$, A)) with 
the (a, F) -split component S, the product map 

U K x M K x U K -)• K 

is bijective and 

s ■ U K ■ s~ l C U K , s" 1 ■ U K - s C U K 

for alls E S-(l). 

(4) For each cr- split parabolic subgroup P = M x U of G standard 
with respect to (S , A0, A), the family {M D K n } n > enjoys the 
same properties as ([I])-® above for the group M. 

Proof. These are derived directly from the corresponding properties of 
K' n in 14.21 First, note that k belongs to K n if and only if both k and 
a(k) belong to K' n . Now, ((Tj) is obvious. For (j2J), take any k 2 E K n 
and consider their commutator. By l4.2l (l2l). we have k\k%ki k 2 E K' n+1 
and 

a(A; 1 )a(A; 2 ) ( T(A; 1 )-V(A; 2 )- 1 = afafotflq 1 ) E K' n+1 , 

hence kxk^k^k^ 1 E K n+ \. For (j3J), it is sufficient to show the sur- 
jectivity of the product map. Given k E K = K n , decompose k and 
o-(k)- 1 E K' — K' n as 

k = u^miUi, a(k)~ 1 = u 2 m%Ui 

where u~ E U^,, m-i E M^, E Uk 1 (i = 1,2) by 14.21 Q. Then we 
have 

a(u 2 )ui = cr(m 2 ) _1 o~(u 2 )~ 1 u^ 1 m^ 1 E U~ D P = {e}, 

which shows that % = (xf^) -1 and in turn, mj = o"(m 2 ) _1 , U\ = 
(7(1*2) -1 - Now we have 

u - = a(u 2 )- 1 G (cr n n (a(U) n cr^')) = U-f)K 

and similarly mj E M fl if, «i E U K. Finally, (jlj) follows once 
(HD-© are verified. □ 

We say that a family {K n } n > of cr-stable open compact subgroups 
of G is adapted to (S , A0, A) if it satisfies the above properties dH) (J^J) . 
By the bijectivity of the product map of ([2D, any element k E K = K n 
can be written uniquely as 

k = u~ ■ m ■ u + , u~ E Uk, m E M K , u + E Uk- 
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Such an expression is called the Iwahori factorization with respect to 
P. 

4.4. Let p be an odd prime and C a finite abelian p-group. The 
homomorphism a > a 2 of C into itself is bijective. The inverse map is 
denoted by a i— > a 1//2 . Let a be an involution on C. If a G C satisfies 
the condition <r(a) _1 = a, then 6 = a 1 / 2 is an element of C such that 
a = bo-ib)- 1 . 

4.5. Lemma. Let p be an odd prime, K a totally disconnected compact 
group and a an involution on K . Suppose that K has a decreasing 
sequence 

K = K ! D K 2 D AT 3 D • • • D AT n D • • • 

o/ a -stable open normal subgroups such that 

(i) K n /K n+ i is a finite abelian p-group for each n ^ 1, 
and 

(ii) f] n ^K n = {e}. 

Then, for any k G K satisfying the condition o(k)~ l = k, there exists 
an element k' G K such that k = k'a(k') . 

Proof. Our discussion below is similar to that of |PRj Theorem 6.8]. 
Look at the involution on the quotient K\jKi induced by a. If a given 
k G K = K\ satisfies <r(A;) _1 = k, then by I4.4[ there exists an element 
y± G K\ such that 

y 1 a(y i y 1 = k mod K 2 . 
Set k<i = y^ x ka(yi). It is an element of K 2 and satisfies 

o-(k 2 y l = Vi 1 a(k)~ l a{yi) = y^kafa) = k 2 . 

Looking at the involution on K 2 /K 3 induced by a, there exists an 
element y 2 G K 2 such that 

V2<y{y2Y 1 = k 2 mod K 3 . 

Set k 3 = y 2 1 k 2 o{y 2 ) G K 3 . We have cr^) -1 = k 3 . In this way, we can 
take k n+ i, y n +i G K n+i from k n , y n G K n by the rules 

k n+1 = y~ l k n a{y^), y n+1 a(y n+1 )~ 1 = k n+1 mod K n+2 . 

Consider the sequence {z n } in K defined by z n = y x y 2 • ■ ■ y n . It has a 
subsequence {z nv } which converges to an element, say k', of K. Note 
that 

k n+1 =y~ 1 --- y^kaiyx) ■ ■ ■ a(y n ) = z~ x ka{z r ), G K n+1 . 

Thus 

(k')- 1 ko-(k')ef)K n „ +1 = {e}, 



16 



SHIN-ICHI KATO AND KEIJI TAKANO 



which shows the claim. □ 

4.6. Lemma. Let P = M x U be a a -split parabolic F -subgroup which 
is standard with respect to (So,Ag,A) and K = K n (n ^ 1) a a- 
stable open compact subgroup of G from the family {K n } adapted to 
(So, A0,A). Then, 

U K C HM K U K . 

Proof. For a given u G Uk, consider the element k := u~ 1 o~(u) G K. 
By using the Iwahori factorization with respect to P, express k as 

k = u~ ■ m ■ u + , u~ G Uk, m G Mk, u + G Uk- 

Since a(k)~ x = k, we have 

a(u + )~ l ■ o(m)~ l ■ o~(u~)~ l = u~ ■ m ■ u + 

so that u + = o"(u - ) -1 and a(m)~ 1 = m. Here note that the group Mk 
satisfies the assumption of Lemma 14.51 Thus we can take an element 
m! G Mk such that m = m' a{m')~ 1 . As a result, we have 

u~ l a{u) = k = u~ ■ m'aim') -1 ■ a(u~)~ l = u~m' a{u~m')~ l . 

This shows that uu~m! G H, hence u G HMkU k . □ 

5. Invariant linear forms on Jacquet modules 

For a given if-invariant linear form A on an ^-distinguished rep- 
resentation, we shall construct a canonical linear form r\p(A) on the 
Jacquet module along each er-split parabolic F-subgroup P by using 
Casselman's canonical lifting. It turns out that rp(A) is M D H- 
invariant where M = P fl cr(P). We have a useful relation between 
(H, A)-matrix coefficients and (M fl H, rp( A) )-matrix coefficients on 
the (a, P)-split component of P. 

5.1. From now on, we say briefly that P is a er-split parabolic subgroup 
of G if it is the group of P-points of a er-split parabolic P-subgroup P 
of G. Also we say that S is the (cr, P)-split component of P if it is the 
group of P-points of the (cr, P)-split component S = Si of P = Pj, and 
so on. As a Levi subgroup of a a-split parabolic subgroup, we always 
take the er-stable one as in 12. 31 

Let (it, V) be an admissible representation of G and P = M \xU a a- 
split parabolic subgroup with the (cr, P)-split component S. We regard 
P as a standard one with respect to a suitable choice of (So, A0, A). 
Let (ttp, Vp) denote the normalized Jacquet module of (it, V) along P: 
The space Vp is given as the quotient V/V(U) where V(U) denotes 
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the subspace of V spanned by all the elements of the form tt(u)v — v, 
v G V, u G U. The action 7ip of M on Vp is normalized so that 

np(m)j P (v) = 5 p 1/2 (m)jp(Tr(m)v) 

for m G M and v G V. Here jp '■ V —>■ Vp denotes the canonical 
projection and Sp the modulus of P. 

5.2. Let us recall the construction of Casselman's canonical lifting. 
For a compact subgroup K of G, let V K be the subspace of V of all 
K -fixed vectors and Vk '■ V — > V the projection operator given by 

V K {v) = ) [ n{k)vdk (v G V). 



K 



vol(K) 

For a compact subgroup U\ of U, set 



n(u)vdu = 

C/i 



It is known that V(U) is the union of all V(Ui) where U\ ranges over 
all compact subgroups of U. Now, for a given v G Vp, take an open 
compact subgroup K = K n from the family adapted to (S o ,A0,A) 
such that v G (Vp) K . Next choose an open compact subgroup U\ of 
U with V K fl V(U) C V(£/i). Finally we take a positive real number 
e ^ 1 such that sf/is -1 C £/# for all s G by Lemma [2781 Since 

S~(e) is contained in we may replace A~(e) in the argument of 

[U| §4] by ^^(e). We have an isomorphism 

V K Hs)V k ) ^ [V p ) Mk 

by the restriction of the canonical projection jp : V — > Vp as in [CJ 
4.1.4]. The element f G (7r(s)V x ) satisfying jp(f ) = v is called t/ie 
canonical lift ofvE Vp with respect to K. It depends on the choice of 
K, but not on U and e. If v' is another canonical lift of v with respect 
to K' C then we have 

v > eV M « u «, v = V K {v')=Vu K W) 

by P 4.1.8]. 

5.3. Proposition. Lei A be an H -invariant linear form on an admissi- 
ble representation (tt, V) of G and P a a -split parabolic subgroup stan- 
dard with respect to (So, A0, A). 

(1) For K = K n (n ^ 1) in the family {K n } adapted to (So, A0, A) 
and v G V MkU k , one has 

(X,v) = (X,V Uk (v)). 
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(2) For two canonical lifts v, v' G V of a given v G Vp, one has 

(X,v) = (X,v'). 

Proof. (1) Let U\ be an open compact subgroup of Uk which fixes v. 
Then we have 

(\,Vu K (v)) = (A, 1 / ir(u)vdu) 

voL{U K ) J Uk 

vo\(Ui) ^ 

Let us express each Ui G £7^ as Ui = himiU~ where hi G H, rrii G 
and G C/^ by Lemma 14.61 Then this is equal to 

< A , ~ttjt\ Y] n(himiur)v ) = ( A , v ) 
voi(Uk) *-r 

since A is if -invariant and v G V MkV k . 

(2) Assume that v (resp. v') is the canonical lift of v G Vp with 
respect to K (resp. iT'). It is enough to consider the case where 
K' C K. By the remark preceding this proposition, we have 

v , e yM K U ii V =P Uk (v'). 

It follows from (1) that 

(X,v) = (X,V Uk (v')) = (X,v'). 

□ 

5.4. After the above proposition, we can define a linear form rp(A) 
on the Jacquet module Vp along a a-split parabolic subgroup P by 

(r P (\),v) = (\,v) 

where v G V is a canonical lift of Tj G Vp. 

5.5. Proposition. Let X fre an H -invariant linear form on an admis- 
sible representation (it, V) of G and P = M x U a a -split parabolic 
subgroup of G with the (a, F) -split component S. 

(1) For each v G V, there exists a positive real number e ^ 1 such 
that 

(X,n(s)v) = 5 P /2 (s)( rp (\),n P (s)j P (v)) 
for any s G S~(e). 

(2) Assume that X is a linear form on Vp having the following prop- 
erty: For each v G V , there exists a positive real number e ^ 1 such 
that 

(X,7T( S )V) = 5p( S ) 1 / 2 (A,7Tp( S )jp( U )) 
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for any s E S (e). Then A coincides with r P (X). 

Proof. (1) For a given v G V, choose an open compact subgroup K = 
K n from the adapted family such that v G V . Take an open compact 
subgroup Ux of U K with V K H V(U) C V{U X ). Let e ^ 1 be a positive 
real number such that sf/iS -1 is contained in Uk for all s G ^(e:). 
Then, by the Iwahori factorization with respect to P, we have tt(s)v G 
V M * u i so that 

j P (7r(s)u) = jp(Vu K (n(s)v)) = jp(P K (7r(s)v)) 

for all s G On the other hand, since jp(v) G (V p ) Mk and s is 

central in M, we have vrp(s)jp(f) G (Vp) Mk so that 

jp(7r( S )t;) = 5 p /2 {s)k p {s) ]p {v) G (V» m -. 

These relations show that Vk{^{s)v) = Vu K {n(s)v) is a canonical lift 
of bp 2 (s)ir P (s)j P (v ). Thus, by definition we have 

(rp(A),5p( S ) 1 / 2 vrp( S )jp(t;)) = (X,V Uk (tt(s)v)) 

and the right hand side is equal to (A, ir(s)v) by Lemma [531 (1). 

(2) Take an open compact subgroup K = K n from the adapted 
family. Let e be a positive real number such that 

(A,vr( S )t;) = 5p( S ) 1 / 2 (A,7rp( S )jp(t;)) 

for all s G S~(e) and all v G V^. This is possible since V K is finite 
dimensional. We may choose this e so that the space Vk(^(s)V k ) is 
independent of s G S~(e) by [Cj 4.1.6]. Since 7r(s)w G V MkU '< , we have 

(A,tt(s)i;) = (A,^(7r(s)T;)) = (A, Vk(^(s)v)) 

for all s G On the other hand, Vk{^{s)v) is a canonical lift of 

1 /2 

<5p (s)7Tp(s)jp(f ) again, so we have 

(X,V K (7T(s)v)}=5 p /2 (s)(r P (X),7T P (s)jp(v)}. 

As a result, we have 

(X,7Tp(s)jp(v)) = (r P (\),Tc P (s)j P (v)} 

for any s G 5-(e). Since jp(V^) = (V p ) Mk (see jd 3.3.3]), this shows 
that A = rp(A) on (V p ) Mk . Letting K vary in the adapted family, we 
conclude that A = rp(A) on Vp. □ 

5.6. Proposition. Let A be an H -invariant linear form on an admis- 
sible representation (jr, V) of G and P = M x U a a -split parabolic 
subgroup of G. 

(1) The linear form r P (X) on V P is M fl H -invariant. 

(2) The mapping r P : (V*) H — > (V P ) MnH is linear. 
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Proof. (1) For a given m G M fl H, set A = r P (X) o n P {m). By (1) of 
Proposition 15.51 for the vector ir(m)v, we can take < e ^ 1 such that 

1 /2 

(A, 7r(s)7r(m)t/) = 5/ (s)(rp(X),np(s)jp(n(m)v)) 
for all s G S~(e). Since s is central in M, the left hand side is equal to 

(A, 7r(m)7r(s)v) = (X,7r(s)v), 
while the right hand side is equal to 

5 P /2 (s)5 P /2 (m)(rp(X), ir P (s)iTp(m)jp(v)) 

= 5 P /2 (s)(r P (X),TTp(m)7Tp(s)jp(v)} = 5 P /2 (s)(X,7Tp(s)j P (v)} 

by Lemma [2. 91 Thus A has the property of Proposition I5.5I (2), which 
implies that r P (X) coincides with A = rp(A) o 7t P (m). 

(2) For any Ai, A 2 G (V*) H and ci, c 2 G C, it is easy to see that 
c iTp(Ai) + c 2 rp(A 2 ) satisfies the unique property that r P (ciX\ + c 2 A 2 ) 
must have in Proposition I5.5I (2) . □ 

5.7. The group case. Here we note that the mapping rp in the situation 
of II .5! is the well-known one constructed by Casselman. Let G be the 
group Gi x G\ with the involution a which permutes factors. Then 
cr-split parabolic subgroups are those of the form P\ x Pj~ where P\ 
and P{ are mutually opposite parabolic subgroups of G\. Set M\ = 
Pi fl P{ . For an irreducible AGi-distinguished representation tti <8> TTi 
of Gi X Gi, let A be the AGi-invariant linear form on m <8> 7rl defined 
by the canonical pairing of 7Ti and 7ri as in II. 51 Then r PlxP -(A) is an 
(Mi x Mi) fl AG i = AMrinvariant linear form on the Jacquet module 
(7Ti <S> 7Ti)p lX p- — (tti)p 1 ® ( 7 ^i)p-- It is exactly the one given by the 
pairing of the Jacquet modules {^\)p 1 and (7Ti) P - constructed in [U| 
4.2.2]. 

5.8. We study the transitivity of the mappings rp with respect to the 
inclusion of a-split parabolic subgroups. 

Let P = M x U be a cx-split parabolic subgroup of G. It is obvious 
that cr-split parabolic subgroups of M are of the form M fl Q where 
Q is a cr-split parabolic subgroup of G contained in P. Let L be the 
cr-stable Levi subgroup of Q. It is also the cx-stable Levi subgroup of 
MnQ. As is well-known, (Vp) Mn g is isomorphic to Vq as an L-module. 
Fix an isomorphism and identify (Vp) Mn Q with Vq from now on. There 
are induced mappings 

rp-.iV) -> (Vp) , r A / n g : (Vp) -> ((V^) MnQ ) 
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and 

r Q : orf - (v£ LnH = ((v P y MnQ ) 

of invariant linear forms. 

5.9. Proposition. For P and Q as above, one has 

r MnQ ° Tp = r Q . 
In other words, the diagram 

[y*) H _^-> (V F 

(v^) LnH = ((v P y MnQ ) 

is commutative. 

Proof. Let A be an element of (V*) H . We put A = r M nQ (rp(\))- It 
is regarded as an L D if -invariant linear form on Vq. Let Sl be the 
(a, F) -split component of Q. By Proposition 15.51 (1), we can choose a 
positive real number e ^ 1 for each u 6 V such that both 



(A,7T(s)«) 



CV2, 



s)(rp(X),7r P (s)j P (v)) 



and 



(r P (X),'Kp(s)jp(v)) 

= S MnQ( s )( r MnQ (r P (\)) , vr MnQ (s)j Mn Q (j>(t;))) 
hold for any s G S^(e). Identifying Ja/oq (jp( v )) with jq{v), we have 
(A,7r(s)u) = 5p 2 (s)4^Q(s)(A,7rQ(s)j Q (t;)) 

= ^ /2 (s)(^ 7r Q(s)jQ(f)) 

for all s e S^(e). This means that A satisfies the unique property that 
r<g(A) must have in (2) of Proposition 15.51 □ 



6. Characterization of relative cuspidality 



We shall give a characterization of relative cuspidality in terms of 
Jacquet modules along cx-split parabolic subgroups. 
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6.1. Let (it, V) be an if-distinguished admissible representation of G. 
For a given non-zero iJ-invariant linear form A G (V*) H on V, we have 
defined the (H, A)-matrix coefficient <f\,v ( v £ V) by 

Recall that (it, V) is said to be (H, A)-relatively cuspidal if the support 
of ipx,v is compact modulo ZH for all v G V. 

Let Z~ denote the (a, F)-split component of Z. We have an almost 
direct product decomposition Z = Z~ • (Z n H). This implies that 
Z/Z~(Z OH) is finite. Hence (tt, V) is (H, A)-relatively cuspidal if and 
only if the support of if\ yV is compact modulo Z~H for all v G V. 

A large part of this section is devoted to the proof of the following 
theorem. 

6.2. Theorem. Let (tt, V) be an H- distinguished admissible represen- 
tation of G and A a non-zero H -invariant linear form on V . Then, 
(tt,V) is (H, A) -relatively cuspidal if and only if rp{\) = for any 
proper a -split parabolic subgroup P of G. 

First we shall prove the only z/part. The proof of the i/part will be 
given in 16.81 after several preparatory lemmas. 

6.3. Proof of the only if part. Let P = M x U be a proper c-split 
parabolic subgroup of G with the (a, F)-split component S. We regard 
P as a standard one Pj with respect to a suitable choice of (S , A0, A) 
and a a-split subset I C A as in 12.31 Assume that (tt, V) is (H, A)- 
relatively cuspidal. Let K = K n be any member of the family {K n } 
of 14.31 adapted to (So, A0, A). Since V K is finite dimensional, we can 
take a compact subset C of G so that the support of ip\ jV is contained 
in Z~CH for all v G V K . Here let us observe that Z~CH nS + (l) is 
contained in some subset of 5* + (l) which is compact modulo Z~: For 
a given s G Z~CH PI 5' + (l), write s = zch with z G Z~ , c G C and 
h G H. Since h = c _1 z _1 s is fixed by a, we have 

cr(c)~ zs~ x = c z~ x s, 

thus 

{z-hf = ca{c)-\ 

This shows that (z^s) 2 , hence also z~ x s, stays in a compact subset of 
S' + (l). Now we may choose a positive real number e < 1 such that 

Z-CH n S + (l) C {s G 5 I 1 ^ < e^ 1 (a G A \ /)} . 

Since s G S~(e) implies that s' 1 ^ Z~CH D ^(l), we have 

(\,it(s)v) = Lpx^s' 1 ) = 
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for all s G S (e) and v G V K . On the other hand, by (1) of Proposition 
15.51 we may choose e' such that the relation 

(X,n(s)v) = 6p(s)^ 2 (rp(X),np(s)jp(v)) 

holds for any s G S~(s') and v G V . Putting e" = min(e, e'), we have 

(rp(\),n P (s)jp(v)) = 

for all s G S~(e") and v G V . This shows that rp(A) vanishes on 
j'p (V^) = (Vp) Mk . Letting K vary in the adapted family {K n }, we 
conclude that r P (\) = on Vp. □ 

6.4. Now we turn to the converse direction. Let A be a non-zero H- 
invariant linear form on an iJ-distinguished admissible representation 
(ir, V) of G. Assuming that rp(A) = for proper a-split parabolic 
subgroups P of G, we shall investigate the support of (H, A)-matrix 
coefficients of n. First we observe a relation between rp and rq when 
P and Q are conjugate. 

6.5. Fix a data (So, A0, A) as in 12.11 and let Po = Mo x Uo be the 
corresponding minimal cr-split parabolic F-subgroup as in 12.41 For an 
element 7 G (M H)(F), we put m 7 = ■ya( r y)^ 1 G M . Define the 
F-involution cr 7 on G by 

The <7 7 -fixed point subgroup in G coincides with , yH , y~ l , which is 
denoted by H 1 . Note that So is also a maximal (<r 7 , F)-split torus. 
Moreover, any cr-split parabolic subgroup P standard with respect to 
(S , A0, A) is also cr 7 -split. The (cr, F)-split component and the (cr 7 , F)- 
split component coincide for such a parabolic subgroup P. Now we can 
define the mapping 

(6.5.1) r P : (V*) H -< -> {V*) M ^ 

for an .^-distinguished representation (ir, V) by changing cr to o 1 and 
H to H^. 

On the other hand, for 7 and P as above, put Q = 7 _1 P7. It is 
cr-split, but possibly non-standard. If S denotes the (cr, F)-split com- 
ponent of P, then the conjugate 7 _1 S7 gives that of Q. We can define 
the mapping 

(6.5.2) r Q : (V*)^ -> (V^~ lM ^ nH 

for an if -distinguished representation (jr,V). Two mappings (16.5.11) 
and (16.5.21) are related by the isomorphism 7r*(7) : (V*) H — > (V*) Hj as 
follows. 
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6.6. Lemma. For 7, P and Q as above, the relation 

(r P (ir*m),jp(v)) = (rQ(A),jQ(7r( 7 -»> 
holds for every A G (V*) H and v G V . 

Proof. The mapping 7r(7 _1 ) : V — > V sends V(U) isomorphically onto 
V('~f~ 1 U'~f), hence induces an isomorphism 

Trfr- 1 ) : V P = V/V{U) -> ^/r( 7 - 1 [/7) = V Q 
with the relation 

^(T -1 ) (JpO)) = Jq (tt(7 _1 )^) • 

Under this notation, the right hand side of the lemma is written as 
(r Q (A),7r(7- 1 ) (j P (v))). Set A = r Q (A) o tt^- 1 ). We show that A 
satisfies the unique property in (2) of Proposition 15.51 that r P (71-* (7) A) 
must have. For s in the (a, F)-split component S of P, we have 

5 P /2 (s)(\,7c P (s)jp(v)) = ( rQ (\),M^)(5 P /2 (s)n P (s)jp(v))) 
= (^(A),^ 1 ) (jp(rr(s)v)) = (r Q (X),j Q (vr^M^))) 
= (^q(A),5q /2 (7" 1 S7)ti"q(7" 1 S7)Jq (tt(7 _1 )^)))- 

By Proposition [53] (1) applied to Q, there exists a positive real number 
e ^ 1 such that the last quantity is equal to 

(A, 7r(7~ 1 s7)7r(7~ 1 )t>) = (7r*(7)A, 7r(s)u) 

for all s G ^(e:). This shows the claim. □ 

Next we shall see what happens if rp(A) vanishes for a single P. 

6.7. Lemma. Fzx a data (So, A0, A) and let I C A fre a a -split subset, 
7 an element of (MoH)(F) and f2 a compact subset of G. Let A 6e 
a non-zero H-invariant linear form on an admissible representation 
(tt, V) of G. Suppose that r 7 -ip J7 (A) = 0. Then for each v G V, 
there exists a positive real number e = Ej n 5: 1 s^c/i £/ia£ tp\ jV vanishes 
identically on QS I ~(e)jH . 

Proof. We abbreviate Pj and Sj(e) respectively as P and S~(e). For 
G fi, s G S~(e) and h £ H, we have 

<P\v{ks lh) = (A, 7r(7~ 1 )7r(s)7r(/i;~ 1 )i;) = (7r*(7)A, 7r(s)7r(A;~ 1 )f ). 

Note that 7c(k~ 1 )v stays in a finite dimensional subspace for any k G Q. 
By applying Proposition 15.51 (1) to the linear form 7r*(7)(A) G (V*) H ~« , 
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we can take a positive real number e ^ 1 such that 

(7r*( 7 )A,7r( S )7r(AT» = 5 P /2 ( S )(r P (7r*( 7 )A), 7r P ( S )jp(7r(Ar»> 

= (rp(7r*( 7 )A),jp(7r( S )7r(A;- 1 )i;)) 

for any s G S~(e) and G f2. By Lemma 16.61 the right hand side is 
equal to 

(r 7 -ip 7 (A), j 1 -i Pl (7r(7" 1 )7r(s)vr(A;^ 1 )f)), 
which is zero by assumption. □ 

Now we give the rest of the proof of 16.21 

6.8. Proof of the if part. Recall from Corollary 13.41 that G is decom- 
posed as 

G = ttS+TH 

for a suitable compact subset Q of G and a finite subset T of (M H)(F). 
Assume that r>(A) = for all proper cr-split parabolic subgroup P. 
For a given v G V, let e be the minimum of Ei a in Lemma [6.71 where 
I runs over all proper cr-split subsets of A and 7 runs over V. Then, 
( P\,v\rts- 1 TH = if s G C\ S]~(e) for some cr-split subset I C A. In 

particular, s G 5^(1) cannot be in Sj (e) for all a G A \ A CT if cpx )V is 
not identically zero on Qs^^^TH. Here I a is the maximal cr-split subset 
of A as in ESI Note that s a = s a * (o) for all s G S if a G A \ A CT . As a 
result, the support of (f\ iV is contained in the union of Qs~ l TH where 
s G Sq(1) satisfies e < |s a |p ^ 1 for all a G A \ A CT . However, the set 
{s G S I £ < ^ 1 (or G A \ A CT )} is finite modulo Z~ ■ S (O F ). 

Hence the support of tp\ )V is compact modulo Z~H. 

This completes the proof of Theorem 16.21 □ 

Recall that (tt, V) is said to be irrelatively cuspidal if it is (H, A)- 
relatively cuspidal for all A G (V*) H . Now we have obtained one of our 
main theorem. 

6.9. Theorem. An H- distinguished admissible representation (it, V) of 
G is H -relatively cuspidal if and only if rp((V*) H ) = for any proper 
a -split parabolic subgroup P of G. 

6.10. Remark. If all cr-split parabolic F-subgroups are if-conjugate to 
one in a fixed standard class (see Lemma \2. 51 (2) for example), then by 
Lemma \6. 61 it turns out that rp((V*) H ) vanishes for all proper cr-split 
P if and only if it does for all standard proper P. Thus, in such a case, 
the relative cuspidality is characterized by the vanishing of rp only for 
all standard proper cr-split P. 
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6.11. The group case. The above characterization of the relative cus- 
pidality gives the well-known theorem due to Jacquet in the group 
case. Let tt\ ® iti be an irreducible AGi-distinguished representation 
of G\ x G\ with the canonical invariant linear form A (see II. 5p . For a 
cr-split parabolic P\ x Pf , the linear form r PiXP -(X) is regarded as the 
Casselman's pairing of (ni)p 1 and (tt\) p - (see 15.71) . It vanishes if and 
only if (vri)p 1 vanishes since Casselman's pairing is non-degenerate [Cl 
4.2.4]. Thus, Theorem 16.91 in the group case asserts that 7i"i is cuspidal 
if and only if {k\)p 1 = for all proper parabolic subgroup Pi of G±. 

7. Relative subrepresentation theorem 

Here we give a proof of the relative version of Jacquet's subrepre- 
sentation theorem. From an inductive argument using the Frobenius 
reciprocity and the transitivity of the mapping rp, it is a natural con- 
sequence of the characterization theorem in the previous section. 

7.1. Theorem. Let it be an irreducible H -distinguished admissible rep- 
resentation of G. Then there exists a a-split parabolic subgroup P = 
MkU of G and an irreducible M TlP -relatively cuspidal representation 
p of M such that it is a subrepresentation oflndp(p). 

Proof. We show this by induction on the dimension r of the maximal 
(a, P)-split tori of G/Z. If r = 0, then G/ZH is compact by [liWl 4.3]. 
Hence every P-distinguished representation is P-relatively cuspidal. 
Assume r > 0. If (7r, V) is P-relatively cuspidal, there is nothing to 
prove. If not, then there exists a non-zero P- invariant linear form 
A G (V*) H such that it is not (P, A)-relatively cuspidal. It follows from 
Theorem 16.21 that there exists a proper cr-split parabolic subgroup P 
of G such that rp(X) ^ 0. Let Q = Mq x Uq be minimal among such. 
Then by Proposition 15.91 and Theorem I6.2| it is seen that the Jacquet 
module tiq is (Mq D P, rg(A))-relatively cuspidal. Apply Proposition 
11.111 and take an irreducible Mq fl P-distinguished quotient p' of ttq. 
By the Frobenius reciprocity asserting that 

Hom G (7r,Indg(p')) -Hom MQ (7r Q ,p') (^0), 

we have an embedding of it into Indg(p'). Now the dimension of maxi- 
mal (<r, P)-split tori in Mq/Aq (where Aq denotes the P-split compo- 
nent of Mq) is strictly less than r. By the induction hypothesis applied 
to p', there exists a a-split parabolic subgroup Mq n P = Mp x Up 
of Mq and an irreducible Mp fl P-relatively cuspidal representation p 
of M P such that p' is a subrepresentation of Ind^ nP (p). Here P is a 
cr-split parabolic subgroup of G contained in Q and M P is the a-stable 
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Levi subgroup of P (see 15 . 8[) . As a consequence, n is a subrepresenta- 
tion of Indg (lnd^ nP (p)) ~ Ind£(p). □ 

7.2. T/ie group case. We shall apply the above theorem to the group 
case. For any irreducible admissible AGi-distinguished representation 
tti eg) 7?i of Gi x (?!, there exist a a-split parabolic subgroup Pi x P{ 
and an irreducible AMi-relatively cuspidal representation p\ ® pi of 
Mi x Mi such that 7Ti <8> 7Tl can be embedded in 

IndJ x x ^(pi ® pi) ~ Indg(pi) ® Ind^(pi). 

Here pi is an irreducible cuspidal representation of Mi. The embed- 
ding 7Ti Indp^pi) on the first factor asserts nothing but Jacquet's 
subrepresentation theorem (see [01 5.1.2]). On the second factor we 
have the embedding tt! Indpi(pl) at the same time (see [Sj 3.3.1]). 



8. Examples of relatively cuspidal representations 

This section is devoted to several examples of relatively cuspidal 
distinguished representations. 

8.1. Proposition. Cuspidal H -distinguished representations are Ir- 
relatively cuspidal. 

Proof. This is immediate from Theorem 16.91 □ 

Certain examples of cuspidal distinguished representations were con- 
structed by Hakim and Mao [HM1, HM2J for the symmetric pairs 
(GL n (E),U n (E/F)) and (GL n (F),O n (F)). 

In the rest of this section, we shall give two examples of non-cuspidal 
but relatively cuspidal distinguished representations. 

8.2. The symmetric pair (C7L n , GL n _i x GLi). For n ^ 3, it is 
known that irreducible cuspidal representations of GL n (F) do not have 
any non-zero GL n _i(F)-invariant linear form (see [0 Proof of Theorem 
2]). Therefore irreducible cuspidals cannot be GL n _i(F) x GLi(F)- 
distinguished. We shall construct a class of relatively cuspidal repre- 
sentations below. For n = 3 similar representations were treated by 
D. Prasad in [P]. Representations attached to finite symmetric pair of 
this type was studied in |vDj . 
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8.2.1. Let G be the group GL n / F . Put 

We consider the inner involution a = Int(e) on G. Let H be the a- 
fixed point subgroup in G and H' the fixed point subgroup of the other 
involution Int(e'). We have 

H'={U2) I A G GL n -x, u G GLi). 

By the relation e = rje'r]' 1 , it is seen that H = rj^H'r]. As a maximal 
(a, F)-split torus, we shall take 

S = {diag(s, 1, • • • ,1, s -1 ) | s G GLi}. 

Let A0 be the maximal F-split torus consisting of all the diagonal 
matrices and A the set of simple roots of (G, A0) corresponding to 
the Borel subgroup of upper triangular matrices. If ej G X*(A$) (for 
1 ^ i ^ n) denotes the F-rational character of A0 given by 

et (diag(ai, a 2 , • • • , a n )) = a i} 

then A is described as 

A = {d - e i+ i I 1 ^ i ^ n - 1}. 

It is easy to see that A is a a-basis. We have 

A CT = {e 2 - e 3 , e 3 - e 4 , • • • , e n _ 2 - e n _i } 

and 

cr{ei - e 2 ) = -(e 2 - e n ), (r(e n _i - e n ) = -(ei - e n _i). 

There is only one proper standard cx-split parabolic subgroup Po, the 
minimal one corresponding to A CT . It is the standard parabolic sub- 
group of type (1, n — 2, 1), i.e., 

Po = Pi.n-2,1 = {( a A*^ eG\a,be GL U A e GL n _ 2 }. 

The cr-stable Levi subgroup Mo of Po is given by 

M = { (% °) G G I a,b G G^, A G GL n _ 2 }. 

Note that a acts on Mo as 

fa o\ /& o\ 
cr A = A , 
\0 b/ VO a/ 

hence M n H H consists of all the matrices of the form ( a a ° ). This 

implies that the Galois cohomology of Mo H H over F is trivial. So 
we have (M H) (F) = M H and thus every proper cx-split parabolic 
subgroup of G is if-conjugate to Pq in this case by Lemma [231 (2). 
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8.2.2. Let Q = P n -2,2 be the standard parabolic subgroup of G of 
type (n - 2,2), i.e., 

Q = {(o£) | A e GL n _ 2 , g e GL 2 ). 

This is not cx-split nor cr-stable, but the conjugate pQp -1 is a-stable. 
Take a representation (p,W p ) of GL 2 (F) and consider the normalized 
induction 

7i = Indg(l GLii _ 2(F) ® p). 

8.2.3. Proposition. J/ p zs an irreducible cuspidal representation of 
GL 2 (F) with trivial central character, then the induced representation 

7T = Indg(l GLn _ 2(F) <g)p) 

is irreducible, H -distinguished and H -relatively cuspidal. 



The irreducibility follows from the general result of [Z] 3.2, 4.2]. 
Up to 18.2.51 we show that 7r carries a non-zero if'-invariant linear 
form (hence is if -distinguished by the relation H = p" 1 ^/ 7 //). Relative 
cuspidality will be seen in 18.2.61 

8.2.4. Set O = QH'. This is a closed (Q, if')-double coset of G since 
Q is Int(e')-stable (see [HW1 13.3]). Let I(p; O) be the space of all 
locally constant mappings : O — > W p satisfying 

*((oS)*) = 5p(^) 1/2 p(^)0(x) 

= |det(A)|-|det(^)|-("- 2 )/ 2 -p(^(x) 

for all (o* g ) E Q and x e O. This is a smooth if'-module by the 

right translation. The restriction map Indg(lGL„_ 2 (F) ® p) — * I(p; C) is 
a surjective fT-morphism. Taking if'-invariants in the dual mapping, 
we have an injection 

(i( p; o)Y^{^(i GLn _ 2{F) ®py) H '- 

So it is enough to construct a non-zero element of (I(p; 0)*) H . 

8.2.5. Set R = Q (1 H' and p R = ((1 G l„_ 2 (f) ® p)5q 2 ) Explicitly 
the subgroup R is given by 

* o \ 

/? = < I A • I Ae GL„_ 2 (F), a,de F> 
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and the representation p R of R on W p is given by 




p( a ° d : 



It is seen that <J)\h> belongs to the unnormalized induction ind^ (p R ) 
for all G l(p;0). Furthermore, the mapping i— > gives an 

isomorphism 

I(p ; e>)^indf'(p«) 
of if'-modules. By a version of Frobenius reciprocity [Cj 2.4.3], we 
have isomorphisms 

(I(p; ~ Hom F , (indj' (p R ), c) ~ Eom R (p R , C Sr ) 

where C$ R is the one dimensional representation of R defined by the 
modulus character 5r of R. Note that 5 R is given by 

* o \ 

det(A)| • |a|- (n " 2) . 




So the space Hom R (p R , C$ R ) consists of all the linear forms v : W p — > C 
satisfying 

(,,p(g») tI -)=|ar«"- 2 " 2 .|< i | l "- 2 " 2 (,,u>) 

for all u> G W p and a, d E F x . Under the assumption that p is cuspidal 
with trivial central character, we can construct such a non-zero linear 
form v : W p — > C by using the Kirillov model JC p of p (see |JL] ). The 
space fC p is C^°(F X ), on which the action of the diagonals is described 
as 

(p(8?)0 (x)=t(ax) 
for £ G /C p , x E F x . Let i/ be the linear form on K, p defined by 

(u,o= [ \x\ (n - 2)/2 ax)d x x 



where d x x denotes a Haar measure on the multiplicative group F x . It 
behaves under the action of ( q S ) = (oS) ("^ °) as 



Ixa-^l^^^^x = \ar (n - 2)/2 ■ \d\ (n - 2)/2 (v,0. 



This is the desired property. 
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8.2.6. Finally we show that it = Ind^lc^.-,^) ® P) is -^-relatively 
cuspidal. Assume the contrary. Recall that any proper cr-split para- 
bolic subgroup is if-conjugate to Pq. By Theorem 17.11 there exists an 
irreducible Mo D if-distinguished (relatively cuspidal) representation, 
say xi ® 6 <E> Xi-, of M ~ F x x GL n _ 2 (F) x F x such that 7r can be em- 
bedded in Indp o (xi®#®X2)- We must have 6 = lcL n - 2 (F) and X2 = xr 1 
by the Mo D i7-distinguishedness. Now we have an embedding 

Indp n _ 2 2 (l G i„_ 2 (F) ® p) ^ Ind P lin _ 2jl (Xi ® lcL„- a (F) ® xr 1 )^ 
which contradicts to the cuspidality of p. 

8.3. The symmetric pair (GL 2n , Sp n ). This kind of symmetric pair 
was studied by Heumos and Rallis in [HRj (cf. [BKSJ for the finite field 
case). They have shown that S'p n (F)-models and (non-degenerate) 
Whittaker models of G = GL 2n (F) are disjoint. It turns out that there 
is no irreducible cuspidal (^-distinguished representation, since 
cuspidals always have Whittaker model. In the following, we shall see 
that irreducible S'p n (F)-distinguished Langlands quotient representa- 
tions of GL 2n (F) constructed in |HR[ 11.3] are Sp n (F)-relatively cusp- 
idal if the inducing representations of GL n (F) are cuspidal in the usual 
sense. 

8.3.1. Let G be the group GL 2n /p and J n G G the alternating matrix 
given by 

/-°io \ 



1 
-1 



Consider the involution o on G defined by 

°(g) = Jug' 1 J' 1 - 

The o"-fixed point subgroup H of G is the symplectic group with respect 
to J n . As a maximal (a, _F)-split torus we take 

S = {diag(si, sx, s 2 , s 2 , ■ - • , s n , , s n ) | s» e GLi). 

Let A0 be the maximal F-split torus consisting of all the diagonal 
matrices and A the set of simple roots of (G, A0) corresponding to the 
Borel subgroup of upper triangular matrices. Then A is a er-basis: We 
have 

Ci — e i+ i if i is odd, 

— (e»_i - e i+2 ) if i is even 

where the notation is as in 18.2.11 The minimal a-split parabolic sub- 
group P corresponding to A CT is the standard one of type (2, 2, • • ■ ,2). 



a(ei - e i+1/ 
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The cr-stable Levi subgroup M of Po is isomorphic to the product of 
n copies of GL 2 . Looking at the action of a on M , it is seen that 
M n H is isomorphic to the product of n copies of SL 2 . Again by the 
triviality of the Galois cohomology of M flH and Lemma [2751 (2), every 
er-split parabolic subgroup is if-conjugate to a standard one. Maximal 
standard u-split parabolics are ones of type (m, 2n — m) where m is 
even and any maximal cr-split parabolics are if -conjugate to such ones. 

8.3.2. Let J' n be the alternating matrix given by 

v - ( M 

Jn ~ \-l n ) 

and H' the symplectic group with respect to J' n . There is an element 
rj G G such that J n = rjJ^T], which gives the relation if = r]~ 1 H'i]. 
Thus an admissible representation (ir, V) of G is if -distinguished if and 
only if it is if '-distinguished. 

8.3.3. Let Q = P n ,n be the standard parabolic subgroup of type (n, n) 
given by 

Q= A,DeGL n }. 

The conjugate ?7Q?7 _1 is cr-stable. Take an irreducible admissible rep- 
resentation p of GL n (F) and form the normalized induction 

i p = Indg (p|det(-)| V2 ®p|det(-)|~ 1/2 ) . 

In |HRl 11.3.1.2], it is shown that I p is if'-distinguished, reducible 
of length 2 and the unique irreducible quotient n p of I p also is H'- 
distinguished (hence if-distinguished). 

8.3.4. Proposition. If p is an irreducible cuspidal representation of 
GL„ (F), then tt p is H -relatively cuspidal. 

Proof. By I6.10I and the remark at the end of I8.3.11 it is enough to 
see that rp ((^ P ) H ) = for all maximal standard cx-split parabolic 
subgroup P. We show that (tt p )p cannot be M n if-distinguished for 
any maximal u-split parabolic P = P m ,2n-m = M x U. Except for the 
case that m = n and n is even, P is not associated to Q, hence (I p )p 
(and also (n p )p) vanishes by the cuspidality of p (see \BZ\ 2.13(a)]). If 
n is even and m = n (namely P = Q), then (I p )p (and also (tt p )p) is 
cuspidal by [BZ, 2.13(c)]. In this case, M = GL n (F) x GL n (F) and 
M fl H = Sp n (F) x Sp n (F), so the cuspidal M-module {it p )p cannot 
be M fl if -distinguished by the remark at the beginning of 18.31 □ 
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